BBM 415 - Fundamentals of Image Processing Laboratory

Images of the Russian Empire: Colorizing the Prokudin-Gorskii Photo Collection

Özhan Taşdemir / 21827866

Introduction

From 1907 to 1915, Sergei Mikhailovich Prokudin-Gorskii, a Russian chemist and photographer, travelled around Russia documenting the Russian Empire with colour photography. He was able to do this on black and white film, by taking three exposures of the same scene with red, green, and blue filters. The three photographs can then can be projected through red, green, and blue filters which superimposes into a single, fully coloured photograph when aligned correctly.

Problem Definition

The glass negatives provided by Library of Congress. The three images correspond to the blue, green, and red filters, respectively. At a high level, our program needs to extract the three color channel images, place them on top of each other, and align them so that they form a single color image. We can then apply additional image processing techniques to produce a vibrant but realistic image with as few visual artifacts as possible. We will go through each of these steps in detail.

Image Alignment

To create colored images, the three channels (Red, Green and Blue) must be aligned correctly. Since the original images are stacked on top of one another and have artifacts, we required to do some preprocessing align images correctly.

This is my step by step approach:

Dividing

Divide tensor to 3 equal parts and get B, G and R channels. In the original images ordering top to bottom B, G, R

Results of stacking 3 channels without alignment

image.png image-2.png image-3.png

Cropping

Images has a artifacts on edges. This creates problems for correct alignment. For a proper alignment I've cropped the borders to get better results

Alignment

Align the cropped red and green channels to the blue channel to get (x, y) displacements for both channels. There are many algorithms that can be used to achieve this, and will be discussed below. Apply the displacements onto the uncropped versions of their corresponding channels. The channels without their borders cropped is used instead of the cropped versions to preserve as much of the image as possible. Merge the three channels together into one final image and save.

I choose Normalized cross-correlation (NCC) to solve the misalignment issue. The NCC is calculated as:

def ncc(a, b):

#normalized cross-correlation: (a./||a|| dot b./||b||)
return ((a/np.linalg.norm(a)) * (b/np.linalg.norm(b))).ravel().sum()

Result of Normalized cross-correlation alignment

image.png image-2.png image-3.png

Most of the photos are correctly aligned with NCC algorithm. But we can clearly see photo of the Emir(7th photo) is not aligned correctly. Comparison in colour intenstities has its limitations, and does not work in some cases. Edge detection does a comparison between the changes in intensities instead. There are many approaches to edge detection such as Sobel edge detection and Canny edge detection.

To solve to problem I've used Canny Edge Detection align images

The Canny edge detector is an edge detection operator that uses a multi-stage algorithm to detect a wide range of edges in images. It was developed by John F. Canny in 1986. Canny also produced a computational theory of edge detection explaining why the technique works. Edge detection is also helpful in aligning images that have a lot of colour warping.

Result of edge detection on Emir

image-4.png

Enhancements & Problems

!!! I have red-green color blindness. For this reason, some of my determinations may be wrong or incomplete. !!!
Image 1

Original image has a deformations. I think we can't solve the problem using classical image proccessing techniques. Deep learning can help this kind of situations

image.png

Image 3

Image 3 looks super greenish and don't has a any color depth. Histogram equalization adds its color depth but does not produce a perfect result. If we had a higher quality image we can perform better results

Evolution of Image 3 (No Proccessing/Edge Alignment/After Enhancement)

image-4.png image-2.png image-3.png

Image 5

I think light comes through to left side of the image. This makes a left side more brighter. Histogram equalization solves the problem

Evolution of Image 5 (No Proccessing/Edge Alignment/After Enhancement)

image-5.png image-6.png image-7.png

Image 6

Autocontrast helps Image 6 to get a warmer look and color

Evolution of Image 5 (No Proccessing/Edge Alignment/After Enhancement)

image-22.png image-8.png image-9.png

Image 9

In image of the lady image dont have a depth. Lady is clear but background is overbright. Histogram equalization helps us to see big picture

Evolution of Image 9 (No Proccessing/Edge Alignment/After Enhancement)

image-18.png image-10.png image-11.png

Image 15

Image 15 suffers from the gamma. Grasses are not vivid much. Gamma correction makes small improvement. Now grasses looks vivid

Evolution of Image 15 (No Proccessing/Edge Alignment/After Enhancement)

image-19.png image-12.png image-13.png

Image 26

Problem with the seller is image looks washed and some areas are super dark. Histogram equalization gives some its colors and solves the darkness problem

Evolution of Image 26 (No Proccessing/Edge Alignment/After Enhancement)

image-20.png image-14.png image-15.png

Image 29

Actually Image 29 looks great but it has cold feeling. Looks blueish. Autocontrast rescues the image from this bad feeling

Evolution of Image 29 (No Proccessing/Edge Alignment/After Enhancement)

image-21.png image-16.png image-17.png